Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create Networking Service #49

Merged
merged 38 commits into from
Dec 3, 2019
Merged

Create Networking Service #49

merged 38 commits into from
Dec 3, 2019

Conversation

ec2
Copy link
Member

@ec2 ec2 commented Nov 27, 2019

mDNS and Gossipsub are in there. Can also pass in a Libp2pConfig struct to start the Network service with a set of bootnodes, pubsub topics, and ip/port for listening. Remaining is adding Kademlia. Going to make adding Kademlia a separate PR since this is getting quite big.

Closes

@ec2 ec2 added Status: DO NOT MERGE Network Libp2p and PubSub stuff labels Nov 27, 2019
@ec2 ec2 changed the title [WIP] Create Networking Service Create Networking Service Nov 27, 2019
@ec2 ec2 self-assigned this Nov 27, 2019
Copy link
Contributor

@austinabell austinabell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will test functionally later, could you also provide some instructions on usage and functionality?

node/src/main.rs Outdated Show resolved Hide resolved
node/src/main.rs Outdated Show resolved Hide resolved
node/src/main.rs Show resolved Hide resolved
node/ferret-libp2p/Cargo.toml Outdated Show resolved Hide resolved
node/Cargo.toml Show resolved Hide resolved
vm/types/Cargo.toml Outdated Show resolved Hide resolved
vm/types/Cargo.toml Outdated Show resolved Hide resolved
vm/types/src/actor/mod.rs Outdated Show resolved Hide resolved
vm/types/src/lib.rs Outdated Show resolved Hide resolved
vm/types/src/message.rs Outdated Show resolved Hide resolved
node/ferret-libp2p/src/service.rs Outdated Show resolved Hide resolved
node/Cargo.toml Outdated Show resolved Hide resolved
Copy link
Contributor

@dutterbutter dutterbutter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be helpful to have more commentary throughout. Additionally, is there a plan to add tests in the future?

Comment on lines +14 to +18
#[behaviour(out_event = "MyBehaviourEvent", poll_method = "poll")]
pub struct MyBehaviour<TSubstream: AsyncRead + AsyncWrite> {
pub gossipsub: Gossipsub<TSubstream>,
pub mdns: Mdns<TSubstream>,
#[behaviour(ignore)]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am curious what #[behaviour(ignore)] and #[behaviour(out_event = "MyBehaviourEvent", poll_method = "poll")] do? Can you explain?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#[derive(NetworkBehaviour)] imparts the NetworkBehaviour trait to MyBehaviour. Then #[behaviour(out_event = "MyBehaviourEvent", poll_method = "poll")] are the attributes that the macro expects, to give it more context.

},
}

impl<TSubstream: AsyncRead + AsyncWrite> NetworkBehaviourEventProcess<MdnsEvent>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be beneficial to have comments for some of these functions below

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These functions are all inherited through a trait. Do you think that it would still need documentation? Also they are not functions that we EVER should call. They are called by Libp2p

Copy link
Contributor

@austinabell austinabell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

quick pass, will look over functionality now

node/ferret-libp2p/src/behaviour.rs Outdated Show resolved Hide resolved
config
.listening_multiaddr
.parse()
.expect("Incorrect MultiAddr Format"),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

function returns a result, are you sure you want this to be a panic?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah. I want it to panic if the user gives an invalid address to listen on.

.parse()
.expect("Incorrect MultiAddr Format"),
)
.unwrap();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here, the function signature is for a result, but there are no error types returned anywhere in the function. If intended to be a panic for these, change the function signature to just return Self

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, you're right. Ill change that to return just Self

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah. I want it to panic if the user gives an invalid address to listen on.

))
.map(|(peer, muxer), _| (peer, core::muxing::StreamMuxerBox::new(muxer)))
.timeout(Duration::from_secs(20))
.map_err(|err| Error::new(ErrorKind::Other, err))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why are you changing the error to an Other io error? Are you sure this is the type of error this should return?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah. We get some sort of libp2p transport error, and i want to convert it to an IO error

node/network/src/service.rs Outdated Show resolved Hide resolved
node/network/src/service.rs Outdated Show resolved Hide resolved
Copy link
Contributor

@austinabell austinabell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be in our best interest to have these functions handle errors instead of panicing at random points unhandled. It may not be an issue now, but it would cause headaches when those panics hit when we are relying on certain services to complete or clean data on other threads.

One way to make the error handling pretty basic is to just have String error types and whenever an error is handled, map the error as such .map_error(|e| format!("<Some error text>: {}", e))?; since the errors don't need specific types or be checked as they are eventually being printed to the console.

One other thing that would be nice is handling sigints/ Ctrl+c, but less important than the error handling.

node/src/main.rs Outdated
Comment on lines 24 to 35
// Create the tokio runtime
let rt = Runtime::new().unwrap();

// Create the channel so we can receive messages from NetworkService
let (tx, _rx) = mpsc::unbounded_channel::<NetworkEvent>();
// Create the default libp2p config
let netcfg = Libp2pConfig::default();
// Start the NetworkService. Returns net_tx so you can pass messages in.
let (_network_service, _net_tx, _exit_tx) =
NetworkService::new(&netcfg, log.clone(), tx, &rt.executor());

rt.shutdown_on_idle().wait().unwrap();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it make sense to have this logic in a run function instead of everything specifically in the main function? imo better for future proofing and if you want to handle Errors better than panicing whenever any error is hit (which I don't think is a pattern we should keep in the future)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Potentially. I feel like its not worth exploring taking this out into a function until we have more services up?

Also w.r.t. errors, the only time we panic is when we can't recover from an error. In node specifically, we unwrap 5 times and expect 2 times. 1 of the unwraps and 1 of the expects is if we cant parse the multiaddr we wanna listen on. Not sure how you would handle that except for panicking. 2 other unwraps is addressed before (has to do with mutex poisioning, which should panic). The other 2 unwraps are if the executor fails to start, which is definitely something we don't know how to handle and should probably never happen. If it happens, your computers messed up probably.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also worth noting is that all of these unwraps and expects (except for the mutex ones) do not happen in random points in time. They will error at node bootup time.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, note on SIGINTs, i will make a PR for that soon 👌

node/src/main.rs Outdated Show resolved Hide resolved
node/network/src/service.rs Show resolved Hide resolved
node/network/src/service.rs Outdated Show resolved Hide resolved
node/ferret-libp2p/src/service.rs Outdated Show resolved Hide resolved
node/ferret-libp2p/src/service.rs Outdated Show resolved Hide resolved
Comment on lines +7 to +17
use tokio::sync::mpsc;

use ferret_libp2p::config::Libp2pConfig;
use ferret_libp2p::service::NetworkEvent;
use network::service::*;

use futures::prelude::*;

use tokio;

use tokio::runtime::Runtime;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one last nit and I'm good :D

@ec2 ec2 merged commit bf60880 into master Dec 3, 2019
@ec2 ec2 deleted the ec2/libp2p-setup branch December 3, 2019 21:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Network Libp2p and PubSub stuff
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants